WORLD INTELLECTUAL PROPERTY ORGANIZATION 
International Bureau 




per 

INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT) 



(51) International Patent Classification 5 

H04N 1/00, 7/00, 7/10 
H04N7/14,H04M 11/00 



Al 



(II) International Publication Number: WO 94/01964 

(43) International Publication Date: 20 January 1994 (20.01.94) 



i 



(21) International Application Number: PCT/US93/05982 

(22) International Filing Date: 18 June 1993 (18.06.93) 



(30) Priority data: 
910,395 



8 July 1992 (08.07.92) 



US 



(71) Applicant: BELL ATLANTIC NETWORK SERVICES, 

INC. [US/US]; 1320 N. Court House Road, Arlington, 
VA 22201 (US). 

(72) Inventor: CRAIG, Bernard, J. ; 10407 Schaeffer Lane, No- 

kesvilie, VA 22123 (US). 

(74) Agent: BECKER, Stephen, A.; Lowe, Price, Leblan & 
Becker, 99 Canal Center Plaza, Suite 300, Alexandria, 
VA 223 1 4 (US). 



(81) Designated States: AT, AU, BB, BG, BR, CA, CH, CZ, 
DE, DK, ES, FI, GB, HU, JP, KP, KR, LK, LU, MG, 
MN, MW, NL, NO, NZ, PL, PT, RO, RU, SD, SE, SK, 
UA, VN, European patent (AT, BE, CH, DE, DK, ES, 
FR, GB, GR, IE, IT, LU, MC, NL, PT, SE), OAPI pa- 
tent (BF, BJ, CF, CG, CI, CM, GA, GN, ML, MR, NE, 
SN, TD, TG). 



Published 

With international search report. 



(54) Title: MEDIA SERVER FOR SUPPLYING VIDEO AND MULTI-MEDIA DATA OVER THE PUBLIC TELEPHONE 
SWITCHED NETWORK 





S 


M 


L 


"SSf 1 


otw 

OCCAM 


VLQC0 MltflAt 




(57) Abstract 

A media server (200, 202) receives, stores and forwards multi-media data and full motion video feature presentations with- 
in a PSTN. The media server (200, 202) receives wide band digital video and MPEG encoded video signals. The wide band digital 
video is encoded using either a real time MPEG encoding for immediate transmission over network facilities to subscribers (100, 
120) or using a multi-pass MPEG encoder for storage by the Media Server (200, 202) for subsequent transmission. The Media 
Server (200, 202) includes DRAM (278), optical and magnetic disk storage (282), high speed tape storage (286) and archival sto- 
rage (290). A Librarian Processor (250) allocates storage space on a media appropriate to retrieval requirements for the video da- 
ta. The Librarian is responsive to a storage Manager Processor (270) and a Session Manager Processor (310) for providing video 
program data to an Output Controller (330). 
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MEDIA SERVER FOR SUPPLYING VIDEO AND MULTI -MEDIA 
DATA OVER THE PUBLIC TELEPHONE SWITCHED NETWORK 



Technical Field 

The invention relates to a data storage 
distribution system using the Public Switched Telephone 
Network (PSTN), and, more particularly, to a media 
5 server for supplying stored image, video and audio data 
to local subscribers over a PSTN. 

Background Art 

Video programming options have experienced rapid 
growth with advances in communications technology. 
10 Recent trends have been directed to user definition of 
programming, with each system subscriber specifying the 
programming to be provided. Such a system is described 
in related co-pending application Serial No. 
07/766,535. 

15 In addition to conventional full-motion video 

programs, advances in personal computer capabilities 
have created increased demands on data base sources to 
supply information to households and business. 
Corresponding increases in data availability, storage, 

20 retrieval and transmission capabilities have supported 
recent . growth multi-media data presentation of 
information. Multi-media presentations include, in 
addition to conventional text, still-images, animation, 
slow and fast scan video, and graphics format visual 
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presentations. Multi-media also includes sound data 
presented in concert with, or independent of, one or 
more of the listed visual presentations . A multi-media 
presentation can range from the equivalent of an 
- 5 electronic book including plain test in combination 
with still images to full motion video presentations of 
" feature movies. Thus , multi-media spans a wide 

spectrum of data storage, retrieval and transmission 
requirements to present information in various formats 
10 to system users. 

As described, multi-media includes full motion 
video presentations. Various systems have been 
suggested to selectively provide a user with requested 
video programming. These systems are sometimes 
15 referred to as programming-on-demand and impulse rpay 
per view. Monslow et al. U.S. Patents Nos . 4.890 f 3?n 
and 4.995,078. respectively issued December 26, 1989 
and February 19, 1991. These patents disclose a 
television broadcast system for real-time transmission 
of a viewer-chosen program at a viewer requested time 
to the requesting viewer's television receiver. The 
system comprises a program collection, which stores TV 
programs on a video tape. Each program is assigned and 
labeled with a catalog number. 
25 A viewer request is transmitted using a 

conventional telephone over the PSTN to a scheduling 
computer. The chosen program is transmitted at the 
.requested time over a broadband cable for viewing on 
the requesting viewer's television receiver. A control 
unit prevents other households connected to the cable 
from viewing the program. 

Cohen U.S. Patent No. 4,949.187 issued August 14, 
1990 describes a video communication system comprising 
a remotely controlled central source of video and audio 
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data. This system allows home viewers to select from a 
choice of movies or other video and audio data that may 
be transmitted from the central source to be stored at 
the viewer's premises. The viewer can play the 
5 selection at any convenient time of his choosing. The 
central source may store the data digitally. Movies 
that need to be frequently accessed are stored on 
disks. Movies that are accessed only infrequently are 
archived . A distribution system CPU controls the 

10 bidirectional flow of data from both types of drives. 
Several movies can be transmitted simultaneously to 
different users through a plurality of input/output 
controllers. To make use of a single transmission 
media, such as a telephone line, a multiplexer is used, 

15 which may multiplex in the time domain or the frequency 
domain • 

Etra U.S. Patent No. 5,012,334 , issued April 30, 
1991 discloses a video image bank for storing a 
plurality of video image sequences. The bank provides 

20 a visual effects set for video production. The bank 
includes a library of video disks with an associated 
library index. A microprocessor is loaded with the 
library index. Groups of still frames, one from each 
identified image sequence, are displayed and an 

25 operator selects any one of the represented image 
sequences for addition to an edit tape and associated 
edit list prepared by the bank.. 

Ciampa et al . U.S. Patent No. 4,635.136 , issued 
January 6 , 1987 discloses a massive inventory of 

30 labeled images, which are stored in a digital form. A 
TV signal for each image is generated by a video 
camera. Stored digital information defines the route 
for a video camera to each site where a video image is 
recorded . 
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Lambert U.S. Patent No. 4,381 , 522 , issued April 
26, 1983 discloses a cable television system which 
includes a plurality of selectable video signal 
sources having selectable television program material. 
5 The system enables a viewer to select desired program 
material available by telephone at a remote location 
and have the selected material promptly available for 
viewing. A minicomputer stores information on the 
program source location , the status of the selected 

10 program source, the available channels and the 
schedules for each. 

Walter et al . U.S. Patent No. 4,513,390 , issued 
April 23, 1985 discloses an information system for 
storage, retrieval and generation of information from a 

15 plurality of multi-media sources. The system uses a 
plurality of source inputs and a plurality of data 
outputs, all under the control of a common computer 
that is operative along a common data bus . Included as 
source inputs are stored memory data, hard copy, 

20 microfiche and direct operator entered data. 

Pocock et al. U.S. Patents Nos . 4 ,734,764 and 
4,941, 040 , issued respectively on March 29, 1988 and 
July 10 , 1990 disclose cable TV systems for 
distributing pre-recorded video and audio messages, 

25 which comprise a remote storage device. The video is 
presented as still frames, which are stored in a 
multiple node frame store. A video frame has one of 
the scan lines in its video blanking signal uniquely 
addressed. 

30 Nakaiima et al. U.S. Patent No. 4,538,176 . issued 

\ August 27, 1985 discloses a video/audio transmission 

system, which sends video and audio information from 
video and audio files at a center to a subscriber 
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through at least one subcenter having a video/audio 
buffer memory. 

Walter U.S. Patent No. 4,506,387 , issued March 19, 
1985 discloses a programming-on-demand cable system, 
5 which allows any one of a plurality of users to request 
any one of a plurality of video programs from a library 
of programs . The programs are stored in memory devices 
selectable by a host computer at a central data station 
in response to an address signal transmitted from the 
10 user. Upon request of the viewers, the programs are 
transmitted at a high non-real-time rate over a fiber 
optic line network. Optical data is then reconverted 
to electrical data and stored for broadcast to the 
viewers . 

15 McCallev et al. U.S. Patent No. 4 ,829,372 . issued 

May 9, 1989 discloses a presentation player which is 
utilized in a digital, interactive communication 
system accessible to a plurality of subscribers who can 
select any of a plurality of pre-recorded video/audio 

20 presentations for viewing on their TV sets. The system 
includes a large scale data base, which stores digital 
information including still frame images and textual 
information in hard disk memory units . 

Beaulier U.S. Patent No. 4,821,121 , issued April 

25 11, 1989 discloses a still image store system, which 
stores and selectively outputs video image data. A 
random access frame store receives input video data 
converted into digital form and transfers the data to a 
disk store for more permanent storage. Full size data 

30 and reduced size data of the same video image are 
stored. 

Bohn U.S. Patent No. 4,888,6 38 , issued December 
19, 1989 discloses a marketing research system for 
substituting stored TV programs for regularly 
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scheduled, broadcast TV programs. The substitute TV 
programs may be transmitted from a central office via 
telephone lines to households of cooperating panelists 
for storage at the household premises. 
.5 While providing multiple data formats from a data 

i base to a user, these prior art systems do not 
integrate network control functions to provide a high 
speed data link between major nodes narrow band encoded 
data from nodes to remote users. In particular, prior 

10 art systems are not responsive to individual user 
requests for providing high speed data connectivity 
from a data source to a node serving one or more users 
and selectively scheduling and supplying requesting 
users with programming. 

15 Related copending application serial number 

07/766,535 filed September 27, 1991 describes Video-On- 
Demand (VoD) Services that provide video programming to 
subscribers over the public switched telephone network 
(PSTN). A menu of video programming information is 

20 accessible at the subscriber's premises. The 
subscriber may transmit ordering information via the 
PSTN to independent video information providers. 
Video programming may be accessed and transmitted to 
the subscriber directly from a video information 

25 provider (VIP) or through a video buffer located at a 
central office (CO) serving the subscriber. 

The VIP transmits coded digital video data over 
wideband PSTN supplied connectivity to a central 
office. The video data may be buffered at the central 

30 office for transmission over a POTS line to the 
subscriber. A subscriber may use. either a standard 
telephone instrument over the PSTN or a dedicated 
control device over an ISDN packet network to order the 
video programming. Such a device is located at a 
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television set of the subscriber and permits a display 
of the program menu on the television screen. 

Connectivity between the central office and the 
subscriber for transmission of video data is provided 
5 by an asymmetrical digital subscriber line (ADSL) 
system. ADSL interface units perform frequency 
multiplexing of digital video information with voice 
information to be transmitted to the subscriber and 
support transmission on the ISDN packet data network of 

10 a reverse control channel from the subscriber to the 
central office. 

However, the Video-on-Demand system dees not 
include an integral library of video program material, 
nor does it support integration of several sources of 

15 programming material to provide a multi-media or mixed 
media presentation. The VoD buffer includes limited 
storage capabilities for video and. audio data supplied 
by a VIP. Enhanced functionality is required to 
support multiple program storage and to integrate data 

20 from various data bases to provide composite 
programming required for multi-media and mixed-media 
presentations . 

To support network management, a need further 
exists for a system which dynamically interacts with 

25 network facilities to reconfigure network resources in 
real-time and in response to information requests . A 
further need exists for a library system for accessing 
and retrieving multi-media programming from storage and 
from independent information providers in response to 

30 user requests and transmitting the data over wide 
bandwidth links to a node serving a user. 

Accordingly, an object of the invention is to 
provide a library of multi-media data including video 
programming on demand using components of the PSTN. 
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Another object of the invention is to provide 
subscriber access to multiple sources of combined image 
and textual programming over the PSTN, 

Still another object of the invention is to 
5 provide real-time subscriber control of multi-media and 
video programming delivery. 

A still further object of the invention is to 
provide a selected video program to a subscriber within 
a predetermined short processing interval after 
10 initiation of a request. 

Disclosure of the Invention 

According to one aspect of the invention, a media 
server for supplying multimedia data includes a user 
request processor responsive to a user request for 

15 supplying information request data. A session 
controller is responsive to the information request 
data for supplying information retrieval data and 
output control data. A data storage device is 
responsive to the information retrieval data for 

20 supplying program data. An output controller, 
responsive to the output control data, supplies the 
requested multimedia data. The multi-media data may 
include text, still images, graphics, low and high 
resolution partial and full motion video and audio. 

25 The output controller includes transmission 

circuitry for transmitting the multimedia data to 
remote end users. According to a feature of the 
invention, a public switched telephone network is 
responsive to the output control data from the session 

30 controller for supplying the multimedia data to remote 
end users. The user request data is supplied to the 
user request processor means by the public switched 
telephone network. 



BNSOOCIO: <WO &401964A1_I_> 



WO 94/01964 



PCI7US93/05982 



According to another aspect of the invention, the 
user request processor includes a transmission receiver 
for receiving the user request data from a remote end 
user. The user request data is supplied to the 
5 transmission receiver by a public switched telephone 
network . 

In accordance with another aspect of the 
invention, the data storage device includes a plurality 
of storage media for storing the program data. The 

10 data storage device may include both optical and 
magnetic memories and may further include three 
different storage media having substantially different 
respective information retrieval times. 

According to another aspect of the invention, 

15 session manager includes an output port selector 
responsive to the user request da~a for supplying 
output port assignment data and a memory for storing 
session status data. The memory includes a table for 
storing output port assignment data, input port 

20 assignment data, and program status data. 

According to another aspect of the invention, the 
media server includes a program storage librarian 
responsive to the information retrieval data for 
supplying program priority data of respective program 

25 data, the data storage device responsive to the 
program priority data for storing the program data. 

According to another aspect of the invention, a 
random access storage unit provides storage of the 
program data. 

30 According to another aspect of the invention, a 

program data input controller is responsive to the 
session controller for supplying real-time program 
data to the data storage device. A video data encoder 
receives the real-time program data and supplies 
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encoded video program data to the data storage device 
and/or to the output controller for immediate 
distribution. 

According to another aspect of the invention, the 
5 session controller includes session registers for 
storing dynamic frame location data indicating a 

~ current frame position of the program data. The 
session controller may further include a memory for 
storing a session routing table having program data 

10 routing data and program source data for active users . 

According to another aspect of the invention, the 
session controller is responsive to user address and 
file selection data for supplying network configuration 
request data. The user request processor may include 

15 security processor for storing user access data and, 
responsive to the user access data, selectively 
supplying the information request data to the session 
controller. 

According a feature of' the invention, the user 
20 request processor can detect a user control link 
failure. The user request processor may further 
include an audio response unit for receiving the user 
request data and for supplying audio responses to a 
requesting user and a program catalog processor 
responsive to the user request data for supplying the 
program catalog data to a requesting user. 

According to another aspect of the invention, the 
user request processor means supplies the user request 
data to a multi-media processing means for supplying 
information retrieval data to the data storage means . 

The media file server system handles requests., 
received from a Video Services Gateway and provides a 
video file feed to the customer via the network link. 
In addition to Video-on-Deraand capability, the server 



25 
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supports other user highly interactive multimedia 
applications . The server is a file management system 
responsible for inputting video files from video and 
multimedia information provides storing these files or 
5 passing the real-time data through to the user, keeping 
track of the user's session, handling all interactive 
control requests from the user, and controlling all 
output to the user. 

The system comprises an input controller, Q.931, 

10 X.25, Tl and DS-3 interfaces, a librarian, a storage 
manager, a session manager, an output controller, an 
interactive processor, and a multi-media application 
processor. The librarian provides file functions 
including distribution of video, audio, stills, and 

15 text selections to the session manager £.~d the multi- 
media application processor at the requ st of itself 
and the multimedia application processor. The 
librarian keeps track of the storage location of all 
multimedia selections and catalogs and stores all 

20 incoming files from VIPs . The librarian records access 
history for determining statistical usage and trends . 
This information is used for, among other things, 
designating frequently used features for assignment to 
rapid access highly addressable storage. Various 

25 storage modules of differing capacity and access speed 
are provided. 

Storage format is MPEG encoded digital video, 
received from the VIPs, or Dl digital video, which is 
MPEG encoded by the server processor and stored in MPEG 

30 format. The processor provides appropriate interface 
with intelligent protocol. 

The above and other objects, features and 
advantages of the present invention will become 
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apparent from the following description taken in 
conjunction with the accompanying drawings. 

Brief Description of Drawings 

Figure 1 is a block diagram of a multi-media 
5 programming distribution system including a media 
server according to. the invention integrated, in a 
public switched telephone network. 

Figure 2 is a simplified block diagram of a media 
server according to the invention. 
10 Figure 3 is a detailed block diagram of a media 

server according to the invention. 

Figure 4 is a block diagram of program storage 
facilities of the media server according to the 
invention . 

15^ Best Mode for Carrying out the Invention 

The overall Video-on-Demand and Multi-Media 
distribution service platform shown in Figure 1 uses 
existing components of the Public Switched Telephone 
Network (PSTN). The system supports storage and 

20 distribution of conventional full motion video programs 
("features") such as movies at user request, i.e., 
Video-on-Demand (VoD) . Also supported are Multi-Media 
presentations and processing, including combinations of 
text, still images, and full and partial motion video. 

25 Video may include interactive activities such as 
training applications, games, shopping, etc. 

The Media Server uses compression techniques to 
store video and other multi-media data in memory 
resources controlled by the Media Server for subsequent 

30 forwarding over interoffice facilities. The switching 
facilities are located in Central Offices (COs) serving 
residential customers or subscribers. Loop electronic 
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devices modify the transmission characteristics of the 
local copper loop, to provide required enhancement to 
the PSTN and permit delivery of full motion video 
information and other media. 
5 High data rate types of multi-media data are 

« encoded to conserve system resources. For example, 

analog video information is first converted to a 
digital format using encoding algorithms standardized 
by the International Standards Organization (ISOs) 
10 Motion Picture Experts Group (MPEG). Each title 
comprises video information stored by the Media Server 
as an addressable data file in conventional data 
processing devices functioning as a video library. The 
function of establishing and monitoring connections 
15 linking a video library port transmitting selected 
information with the end user ports receiving the 
information is performed by a supervisory controller 
such as a network control system, e.g., FLEXCOM 
software, used to control the electronic digital cross- 
20 connect switches (DCS) in the PSTN. The DCS, also used 
for switching two-way DS-1 rate transmissions, is 
adapted to additionally provide bridging or broadcast 
of video information to several users . Data session 
control between a video library port and a remote user 
25 one of the functions performed by the session manager 
of the Media Server. 

The network control software of the Media Server 
in combination with management resources of the PSTN 
(i.e., FLEXCOM/LINC or other Network Management System) 
30 control the "network session" between an Output 
Controller 1 of the Media Server, external program 
providers, and user ports. The session manager also 
maintains a record of relevant data regarding each 
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session which is forwarded to a customer billing 
system. 

Customer local loops equipped with Asymmetric 
Digital Subscriber Line (ADSL) devices are connected to 
5 the DCS. The multi-media distribution system provides 
for the simultaneous transport of a one-way # 1.544 
- megabit per second (MBP.S) signal over the same twisted 

pair transmitting voice messages to the residential 
subscriber. The ADSL transported signal is 
10 demultiplexed and the 1.544 portion is then decoded 
using MPEG standard techniques to deliver a full motion 
video signal. In the PSTN , fiberoptic technology will 
replace existing coaxial and twisted pair connectivity 
with corresponding enhancements made to switching and 
15 routing equipment. 

Referring to Figure 1, a multi-media system 
includes CO equipment 10 which is part of a PSTN. CO 
10 provides connectivity from information providers 140 
and 160 through the system to Media Servers 200 and 202 
20 to Multi-Media subscriber 100 and VoD subscriber 120. 

CO equipment 10 includes a conventional voice 
switch 12 which includes means to detect off -hook, 
service requests, call completion ( i.e. , ring 'rip), a 
DTMF decoder 14 and dial pulse detector. Voice switch 
25 12 also includes an actual telephone call connection 
switch for routing voice circuits among the various 
ports. The CO equipment shown may be physically 
distributed over several sites. 

Connected to voice switch 12 are ADSL equipment 16 
30 and 18 for multiplexing (i) voice and signaling 
information from voice switch 12 and (ii) digital 
multi-media data from Digital Cross-Connect System 
(DCS) 24 onto respective subscriber local loops 20 and 
22. Multi-media data from multi-media information 
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providers 140 and 160 is provided ro DCS node 25 where 
it is selectively supplied to Media Server 200 or 
immediately transmitted to DCS node 24 under control of 
Network Management System 28. 
5 In addition to providing pre-stored video and 

multi-media programming, the Media Server accepts video 
and mult i -media programming from video, and Mult i -Media 
Information providers 140 and 160 for later 
transmission and for real-time and multi-pass MPEG 
10 encoding. Real-time encoding is used to provide 
encoded full motion video while minimizing network 
transmission requirements and providing a signal 
compatible with ADSL connectivity to subscriber 
premises 100 and 120. Multi-pass encoding performed by 
15 the Media Server provides a higher quality video signal 
for storage and later transmission over the network to 
subscriber premises 100 and 120. 

ADSLs 16 and 18 multiplex data on subscribers 
loops 20 and 22 using frequency multiplexing, dividing 
20 the available loop bandwidth into three segments. Base 
band audio and signaling below 4 kilohertz (kHz) 
orovides connectivity for a conventional telephone 
services available on the "plain old telephone system" 
(POTS). Alternatively, ISDN channel requirements 
25 consume the bottom 80 kHz of loop bandwidth. Reverse 
channel digital packet information is positioned 
between 80 and 9 0 kHz providing approximately 16 
kilobits per second (kbps) connectivity from the 
subscriber premises to a packet switched network such 
30 as ISDN network over a D-channel interface. Compressed 
video and multi-media data is contained between 100 and 
400 kHz to provide a 1.6 MBPS channel for transporting 
the video and multi -media data over respective loops 20 
and 22 to customer premises 100 and 120. 
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The Media Server is shown in further detail in 
Figure 2 . The Media Server system processes all 
requests from Video Gateway 30 for providing multi- 
media and video file feeds to customer premises via the 
5 DCS and ADSL systems . The Media Server preforms input 
of video and other information files from Video Multi- 
Media Information providers 140 and 160, stares these 
files or passes real-time data through to subscribers 
100 and 120, monitors and records user sessions, 

10 processes all interactive control requests from users, 
and controls all output to the user. 

The Media Server also processes and supplies 
multi-media services including interactive learning, 
interactive games , and other presentations . The Media 

15 Server stores and processes all data required for the 
multi-media applications including text, still 
pictures , audio , and partial and full motion video, as 
well as interactive application scripts. 

The Media Server includes eight major component 

20 systems: Input Controller 210, Q.931 ISDN Interface 
230, Librarian 250, Storage Manager 270, Session 
Manager 310, Output Controller 330, interactive 
processor 350 and multi-media application processor 
370. 

25 Input Controller 210 includes a plurality of DS-3 

Input Ports 212 for receiving multi-media data 
including digitized video in the form of MPEG encoded 
digital video signals and unencoded video. Dl digital 
video is supplied to Librarian 250 for MPEG encoding. 

30 Video supplied in MPEG encoded format is supplied to 
Session Manager 310 for storage by the Media Server or 
for immediate transmission by output controller 330. 

Q.931 Interface 230 provides connectivity between 
Interactive Processor 350 and ISDN D Channel Interface 



BNSOOCID: <WO 9401 964A1 J_> 



WO 94/01964 



17 



PCT/US93/05982 



38 (Figure 1). Interactive Processor 350 receives 
command data over the ISDN from subscribers running 
interactive programming from the Media Server. 
Interactive programs include Multi-media presentations 
5 and video, games. Interactive Processor 350 also 
receives command data from subscriber control unit 130 
for interactive control of feature presentations (i.e., 
conventional video programming including movies, 
concerts, etc.) including pause, fast-forward, reverse, 

10 and other "VCR" type capabilities supported by the 
Media Server. These latter commands are passed to 
Session Manager 310 for further processing and control 
of the video programming. 

Data to be stored or retrieved from memory is 

15 first routed through Librarian 250 and, under its 
control, Storage Manager 270 either stores the program 
data or retrieves and provides previously stored 
program data to Output Processor 330. 

Media Librarian 250 controls distribution of 

20 video, audio, still image data and text selections to 
Storage Manager 270 in response to information requests 
from Session Manager 310 and Multi-Media Application 
processor 370. 

Referring to Figure 3, the Librarian further 

25 monitors and record in Feature Index System 252 the 
storage location of all video selections for VoD and 
Multi-media applications. The Librarian also records a 
history of access to video programming, i.e., 
."features", and to other data provided during each 

30 twenty-four hour period in Usage Data Accumulation 
System 260. The usage data is supplied to Usage 
Probability Processor 262 to establish an intelligent 
cache using DRAM Storage 278 for rapid access and 
highly addressable storage of features. 
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Feature Index System 252 maintain a catalog of 
data and support processing for storing all locally 
stored programs. This includes the allocation of 
storage media type and space, maintenance of addressing 
5 tables for program status and frame indices, and 
maintain an index for all volumes . The program listing 
data is supplied to a menuing system supported by Video 
Gateway 30 (Fig. 1) via Packet Data Network 40. 

Feature Index System 252 performs catalog 

10 maintenance functions including input of new feature 
program data into the system, ageing, and deletion or 
archival of aged program data. New program data from 
Video and Multi-Media Information Providers to be 
stored by the Media Server are received via a DS-3 port 

15 212 of Input Processor 210. The catalog maintenance 
system determines the priority of the incoming program 
data and allocates appropriate storage to the data. 
Once the file space is allocated and the file is 
stored, the volume indexes are updated, the frame 

20 positions of the program are calculated and frame 
addressing tables are created and stored. The frame 
addressing tables are used to address a feature from 
any position in the feature. 

When it is determined that a stored feature is no 

25 longer required, Media Librarian 250 removes the 
program data from the file catalog. The record to be 
deleted is then flagged by a system administrator of 
Storage Allocation Processor 272. 

Librarian 250 tracks frequency of feature access. 

30 As a feature is requested less frequently, it is "aged" 
by the Media Server. Usage Probability Processor 262 
assigns a priority value to the feature which is used 
to determine the appropriate storage type to maximize 
system resources while providing acceptable access time 
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to th- feature based on its demand history. Once a 
feature is aged to a point of not having been requested 
within a predetermined time period, the Media Server 
removes the program from on-line storage units 278 , 282 
5 and 286 and places it in archival storage 290. The 
feature header data remains stored in Feature Index 
252, although access time for the feature will be 
increased . 

The catalog system updates feature titles data as 
10 the features are loaded into the Media Server. The 
updated catalog information is supplied by Librarian 
250 of the Media Server to Video Gateway 30. The data 
supplied to the Video Gateway is shown in Table 1. 
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Alternatively, according to a preferred embodiment 
of the invention, menu data is retained in the Media 
Server and downloaded to user customer provided equipment 
(CPE) for viewing and selection or transmittal to the 
5 user directly for processing. 

Usage Probability Processor 262 statistically 
determines features having the highest probability of 
usage on a per hour and day of week basis to properly 
allocate high order storage, i.e., DRAM Storage 278, on 
10 an hourly basis. In support of this function, Usage Data 

Accumulation Memory 260 stores tables of data including 
time of viewing, day of viewing and cumulative number of 
requests that are updated each time a feature is 
supplied. 

15 Allocation of storage in the Media Server is based 

on the ranking of a feature and the output of the trend 
processing performed by Usage Probability Processor 262. 
Referring to Fig. 4, storage is divided into several 
components, including DRAM 278, Magnetic Tape 290. All 

20 features are stored on the appropriate media based on the 

priority ranking of the feature. For example, DRAM 
Storage 27 8 is used for the highest priority features as 
determined by the trend processing whereas Archival 
storage is used for the lowest priority features. Table 

25 2 gives typical priority assignments and storage 

capabilities of each media. 

PRIORITY NUMBER OF FEATURES STORAGE 

1 20 + 5 DRAM 

2 100 t 3- MAGNETIC 
30 DISK 

3 300 ± 50 HIGH SPEED 

TAPE 

4 500 + ARCHIVAL 

TAPE 

35 TABLE 2 
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DRAM Storage 27 8 is used for the highest twenty 
features and data files as determined by the trend 
processing. When a feature or other high priority media 
file is stored in DRAM it does not occupy magnetic 
5 storage space until removed from DRAM storage. A 

compressed feature length movie of 90 minutes duration 
occupies approximately 1.2 gigabytes of DRAM storage. 

Disk Storage Units 282 are fast access magnetic 
and/or optical media providing storage for the second 
highest fifty priority features and media data files as 
determined by the trend processing. When a feature is 
stored on a Disk unit, it does not occupy DRAM or Tape 
Storage until removed. 

High Speed Tape Storage Units 286 store features and 
15 multi media data files having a medium priority as 

determined by the trend processing. Finally, Archival 
Tape Storage is used for the lowest priority features and 
media data files. 

Storage Channel Management Processor 274 controls 
flow of data between all storage devices and Output 
Control Processor 330. Input Control Processor 210 
receives multi-media and feature program data from Video 
and Multi-Media Information Providers 140 and 160. 
Feature data input streams are accepted either as MPEG 
25 encoded digital video or as Dl digital video to be 

encoded by the Media Server. All data is either stored 
for play at a users request or passed directly through to 
the user as real-time programming. 

MPEG encoded video data received at DS-3 port 212 of 
Input Processor 210 is routed to Session Supervisor 
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312 for storage by the Media Server or as flow-through 
to an appropriate output port 332 of Output Processor 
330. Session Manager 310 supervises the data flow once 
connection from the Video Multi-Media Information 
Provider to the subscriber is established by Input 
Control Processor 210 and Output " Processor 330. No 
data is stored and no index information is supplied to 
Librarian 250 in the case of flow-through data. 

For non-encoded data, a simplified form of MPEG 
encoding is performed by Real-Time MPEG Encoder 390, 
requiring a latency of approximately ten minutes from 
data input to data output. Multi-Pass MPEG Encoder 392 
is used for non-real-time processing, and provides full 
MPEG encoding of video data. Both MPEG encoders 
15 receive non-encoded video data from Input Processor 210 
and provide encoded video to Storage Allocation 
Processor 272 of Storage Manager 270. The encoded 
video data is either stored in an appropriate storage 
media or transmitted by Storage Channel Management 
20 Controller 2 76 to Output Controller 310 for 
transmission to a subscriber over the PSTN. 

Session Manger 310 include Session Supervisor 312 
which manages all program sessions including system 
access by Video-on-Demand subscribers, other Multi- 
25 media users and input and output to, from and through 
the Media server by Multi-Media Information Providers. 
Session Supervisor 312 tracks and records all data 
pertinent to each session including output port, input 
port (if receiving data directly from a Multi-media 
30 Information Provider or if the user is a Multi-media 
Information Provider), feature being played or Multi- 
media application address, feature index data, feature 
frame data, and session condition. The Session 
Supervisor also performs network resource optimization 
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by the simultaneous broadcast of the same feature over 
one output port to multiple subscribers. However, each 
subscriber session is individually managed to permit 
interactive features such as video pause. 

Session Trace Processor 314 tracks each 
subscriber's session based on feature and frame so that 
the execution of a pause sequence can be managed for 
each individual session, A subscriber may start 
viewing a feature as part of a larger group, with a 
subsequent selection of a pause function placing the 
subscriber "out of sync" with the rest . of the group. 
Once out of sync, a separate session is established for 
that user requiring Session Manager '310 to cause the 
Gateway to establish a new network link to the 
15 subscriber's viewing location. Note that, although 
each user is assigned an individual logical session, 
sessions remaining in sync with each other share 
output and network facilities . 

Session Supervisor 312 maintains a matrix of 
routing information for each session, including what 
users are assigned to a given output port, and the 
source of the input data: Input Port number or local 
storage location in use. A file play clock uses the 
total number of frames in a given feature to estimate, 
25 by time, the frame position for that feature. This 
data is provided to the Session Registers 316 for 
tracking frames for each logical subscriber session and 
is used to restart an interrupted session. Session 
Supervisor 312 discontinues register updates upon 
30 detection of a session interruption. The Session 
Manger 312 receives requests for initial session 
establishment and for file retrieval from Gateway 30 
via Q.931 Interface 230 and Control Channel Filter 354. 
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Interactive Processor 350 processes incoming 
commands from subscribers sites once a session is 
established by the Gateway. These commands include 
pause or like feature for Video-on-Demand, and all 
5 response and requests by Multi-Media work stations. 
The incoming data is interrogated for content and 
selectively forwarded to the appropriate module for 
processing: Session Manager 310 for Video-on-Demand and 
other retrieval applications and to Multi-Media 

10 Application Processor 370 for Multi-Media Applications. 

Control Channel Monitor 352 monitors each 
subscriber's control channel to detect a communication 
failure. In the event of a failure, Session Manager 
310 is informed and corrective action is initiated. 

15 Referring to Fig. 1, dial-up access to the Media 

Server in the absence of dedicated Control Unit 130 is 
provided by voice switch 12 which establishes 
connectivity with VRU 34 which answers the incoming 
call. The VRU prompts the subscriber for a user ID 

20 which is input via the DTMF pad of telephone instrument 
122. The user ID is checked and, if valid, the VRU 
prompts for a video selection. The video selection is 
then input using the DTMF pad of telephone instrument 
102 which then sends the selection information to the 

25 Media Server via Video Gateway 3 2 and Packet Data 
Network 40. The Media Server identifies the requested 
title and determines if the title is available. 

If the title is found, and has not been previously 
queued for transmission, the corresponding data file 

30 is opened and a reserve idle communications port is 
identified for transmission of the video data to an 
associated DCS node 24 or 26 ; Transmission of the data 
is delayed for a predetermined number of minutes in 
response to a first request for the video selection to 
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allow for simultaneous transmission of the video data 
file to subsequent subscribers placing an order for the 
same title within the delay period. The subsequent 
request orders are also placed in the queue and the 
5 associated communications port ID is matched to the 
subscriber's network address. 

After expiration of the predetermined delay, i.e., 
when current time equals the designated start time, the 
video data file is transmitted from Media Server 

10 through the DCS to the designated ADSL interface 18 
for transmission to subscriber premises 120. At the 
end of the program, a message is transmitted by the 
Media Server to Network Management System 28 to take 
the system down by terminating DCS connectivity. 

15 In an enhanced version of the Video-on-Demand 

system, subscriber premises 120 (Figure 1) is provided 
with a control unit 130 for receiving data commands 
from remote control 132. Remote control 132 can be a 
conventional infrared remote control for interacting 

20 with control unit 130. Data from control unit 130 is 
provided to subscriber ADSL interface 124 for 
transmission of command data zo ISDN D-channel 
interface 38. The ISDN D-channel interface can provide 
initial subscriber order information directly to video 

25 gateway 30 in lieu of using voice response unit 34. 
Once video gateway 32 identifies a subscriber request 
to establish connectivity with the Media Server, the 
address of control unit 130 is provided to the 
Interactive Processor 350 (Fig. 3). Upon receipt of 

30 the corresponding Interactive Processor, network address 
by control uni~ 130, direct connectivity is established 
between control unit and the Media Server 200 or 202 
over packet data network 40. This connectivity permits 
direct data transfer between the customer premises and 
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the Media Server to support interactive video control 
used in interactive educational programming and 
interactive video presentations such. as video games. 

Control unit 130 also supports interactive control 
5 of the buffered video data by transmitting appropriate 
commands to Session Manager 310. These controls allow 
the subscriber to fast forward, rewind, forward, 
reverse search and pause the video/audio data provided 
via DCS 24 to the subscribers premises. 

10 VRU 34 may additionally include voice recognition 

capability to support voice recognition security 
functions and voice selection of video programming. 

Although the present invention has been described 
and illustrated in detail , it is clearly understood 

15 that the same is by way of illustration an example and 
is not to be taken by way of limitation, the spirit and 
scope of the present invention being limited only by 
terms of the appended claims. For example, MPEG 
encoding may be performed by an attached processor that 
is part of the server architecture. Further, each 
component of the server architecture may reside on a 
separate processing platform and be closely coupled by 
high' speed application program interfaces (APIs). 
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Claims 

1. A media server for supplying multimedia data, 
comprising: 

means for processing user request data, said user 
request processing means being responsive to user 
>5 request data for supplying information . request data; 
• : ~ means for controlling sessions of said media 

server, said session control means being responsive to 
said information request data for supplying information 
retrieval data and output control data; 
10 data storage means responsive to said information 

retrieval data for supplying program data; and 

means for controlling the output of said media 
server, said output control means being responsive to 
said output control data for supplying said multimedia 
15 data, 

2. The media server according to claim 1 wherein 
said output control means comprises transmitting means 
for transmitting said multimedia data to remote end 
users . 

3. The media server according to claim 1 further 
comprising a public switched telephone network 
responsive to said output control data from said 
session control means for supplying said multimedia 

5 data to remote end users . 

4. The media server according to claim 3 wherein 
said user request data is supplied to said user request 
processing means by said public switched telephone 
network . 
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5. The media server according to claim 1 wherein 
said user request processing means comprises 
transmission receiving means for receiving said user 
request data from a remote end user. 

6 . The media server according to claim 5 further 
comprising a public switched telephone network for, 
supplying said user request data from a remote end user 
to said transmission receiving means. 

7. The media server according to claim 1 wherein 
said data storage means includes a plurality of storage 
media for storing said program data. 

8. The media server according to claim 7 wherein 
said data storage means includes optical storage means 
and magnetic storage means . 

9 . The media server according to claim 7 wherein 
said data storage means includes three different 
storage media having substantially different respective 
information retrieval times. 

10. The media server according to claim 1 wherein 
said session management means includes: 

output . port selection means responsive to said 
user request data for supplying output port assignment 
5 data; and 

memory means storing session status data. 

11. The media server according to claim 10 
wherein said memory means of said session management 
means includes table means storing output port 
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assignment data, input port assignment data, and 
5 program status data. 

12. The media server according to claim 1 further 
comprising program storage librarian means responsive 
to said information retrieval data for supplying 
program priority data of respective program data, said 

-5 data storage means responsive to said program priority 
data for storing said program data. 

13. The media server according to claim 1 further 
comprising a random access storage unit for storing 
said program data. 

14. The media server according to claim 1 further 
comprising program data input control means responsive 
to said session control means for supplying real-time 
program data to said data storage means . 

15. The media server according to claim 14 
further comprising video data encoding means receiving 
said real-time program data and supplying encoded video 
program data to said data storage means. 

16. The media server according to claim 14 
further comprising video data encoding means receiving 
said real-time program data and supplying encoded video 
program data to said output control means, 

17. :e media server according to claim 1 wherein 
said session control means includes session register 
means storing dynamic frame location data indicating a 
current frame position of said program data. 



BNSOOCID: <WO 9401964A1J_> 



WO 94/01964 PCI7US93/05982 

30 



18. The media server according to claim 1 wherein 
said session control means includes memory means 

. storing a session routing table having program data 
routing data and program source data for respective 
5 active users . 

19. The,.. media server according to. claim 1- wherein 
said session control means is responsive to user 
address and file selection data for supplying network 
configuration request data. 

20. The media server according to claim 1 wherein 
said user request processing means includes security 
means storing user access data and, responsive to said 
user access data, selectively supplies said information 

5 request data to said session control means. 

21. The media server according to claim 1 wherein 
said user request processing means includes means for 
detecting a user control link failure. 

22. The media server according to claim 1 wherein 
said user request processing means includes an audio 
response unit for receiving said user request data and 
for supplying audio responses to a requesting user. 

23. The media server according to claim 1 wherein 
said user request processing means includes program 
catalog data storage means responsive to said user 
request data for supplying said program catalog data to 

5 a requesting user. 

24. The media server according to claim 1 further 
including a multi-media application processor, said 



BNSOOQO: <WO 9401 964A1 _l_> 



WO 94/01964 PCT/US93/05982 

31 



user request processing means supplying said user 
request data to said multi-media processing means and, 
5 in response, said multi-media processing means 
supplying said information retrieval data to said data 
storage means. 

25.. In a public switched telephone network (PSTN) 
for providing multi-media data to one of a plurality of 
subscriber premises, including a cross-connect system 
responsive to control data for establishing 
5 connectivity between a multi-media data base and said 
subscriber premises, the improvement comprising: 

a media server for supplying said control data and 
supplying said multimedia data, including 

(i) means for processing user request data, said 
10 user request processing means responsive to user 

request data from the PSTN for supplying information 
request data, 

(ii) means for controlling sessions of said media 
server, said session control means being responsive to 

15 said information request data for supplying the control 
data to the cross-connect system and supplying 
information retrieval data, and 

(iii) a multi-media data base responsive to said 
information retrieval data for supplying the multimedia 

20 data to remote end users. 

26. A media server for supplying full motion 
video data, comprising: 

means for processing user request data, said user 
request processing means being responsive to user 
5 request data for supplying information request data; 

means for controlling sessions of said media 
server, said session control means being responsive to 
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said information request data for supplying information 

retrieval data and output control data; 
10 data storage means responsive to said information 

retrieval data for supplying program data; and 

means for controlling the output of said media 

server, said output control means being responsive to 

said output control data for supplying said full motion 
15 video data* 
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(54) System and method for indexing between trick play and normal play video streams in a video 
delivery system 

(57) A system and method for indexing between vid- 
eo streams in an interactive video delivery system. The 
interactive video delivery system includes at least one 
media server which stores video streams having differ- 
ent presentation rates. In one embodiment the system 
stores a normal play stream and one or more corre- 
sponding trick play streams. The trick play video 
streams are fast forward and/or fast reverse video 
streams. The system generates index tables or look-up 
tables between the normal play and trick play video 
streams which enable indexing between the streams, 
and uses these look-up tables to switch back and forth 
between the streams. In creating the index tables, the 
system first analyzes the normal play stream and cre- 
ates a normal play time standard based on presentation 
timestamps from the normal play stream. The system 
then creates an index table or look : up table for each of 
the normal play and trick play video streams using the 
normal play time standard. Each index table includes an 
array of two-tuples, wherein the two-tuples are the nor- 
mal play time standard and an index or offset into the 
respective stream. The index tables enable indexing be- 
tween the streams. During video delivery, the system 
uses the respective index tables to switch back and forth 
between the normal play and trick play video streams. 
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D scriptlon 

BACKGROUND OF THE INVENTION 
Fl Idofth Inv ntlon 

The present invention relates generally to video delivery and video-on-demand systems, and more particularly to 
a video server system and method for indexing between video streams having different presentation rates, i.e., normal 
play, fast forward and fast reverse video streams. 

Description of the Related Art 



Video-on-demand or video delivery systems enable a plurality of users or viewers to selectively watch movies or 
other audio/video sequences which are stored on one or more video servers or media servers. The video servers are 
connected through data transfer channels, such as a broadcast cable system or satellite broadcast system, to the 
plurality of_users or subscribers. The video servers store a plurality of movies or other audiovvideo sequences, and 
each user can select one or more movies from the video servers for viewing. Each user includes a television or other 
viewing device, as well as associated decoding logic, for selecting and viewing desired movies. When a user selects 
a movie; the selected movie is transferred on one of the data transfer channels to the television of the respective user. 
20 Full-motion digital video requires a large amount of storage and data transfer bandwidth. Thus, video-on-demand 

systems use various types of video compression algorithms to reduce the amount of necessary storage and data 
transfer bandwidth. In general, different video compression methods exist for still graphic images and for full-motion 
video, video compression methods for still graphic images or single video frames are referred to as intraframe com- 
pression methods, and compression methods for motion video are referred to as interframe compression methods. 
25 Examples of video data compression for still graphic images are RLE (Run-Length-Encoding) and JPEG (Joint 

Photographic Experts Group) compression. The RLE compression method operates by testing for duplicated pixels in 
a single line of the bit map and storing the number of consecutive duplicate pixels rather than the data for the pixel 
itself. JPEG compression is a group of related standards that provide either lossless (no image quality degradation) 
or lossy (imperceptible to severe degradation) compression types. Although JPEG compression was originally designed 
^o for the compression of still images rather than video, JPEG compression is used in some motion video applications. 

In contrast to compression algorithms for still images, most video compression algorithms are designed to com- 
press full motion video. Video compression algorithms for motion video use a concept referred to as interframe com- 
pression, which involves storing only the differences between successive frames in the data file. Interframe compres- 
sion stores the entire image of a key frame or reference frame, generally in a moderately compressed format. Succes- 
35 sive frames are compared with the key frame, and only the differences between the key frame and the successive 
frames are stored. Periodically, such as when new scenes are displayed, new key frames are stored, and subsequent 
comparisons begin from this new reference point. It is noted that the interframe compression ratio may be kept constant 
while varying the video quality. Alternatively, interframe compression ratios may be content-dependent, i.e., if the video 
clip being compressed includes many abrupt scene transitions from one image to another, the compression is less 
^0 efficient. Examples of video compression which use an interframe compression technique are MPEG, DVI and Indeo, 
among others. 

MPEG Background 

4 $ A compression standard referred to as MPEG (Moving Pictures Experts Group) compression is a set of methods 

for compression and decompression of full motion video images which uses the interframe compression technique 
described above. MPEG compression uses both motion compensation and discrete cosine transform (DCT) processes 
and can yield compression ratios of more than 200:1 . 

A general background to and more information about MPEG can be found in the I SO/1 EC MPEG specification 
50 referred to as ISO/IEC 13818, which is hereby incorporated by reference in its entirety. 

The MPEG standard requires that sound be recorded simultaneously with the video data, and the video and audio 
\ data are interleaved in a single file to attempt to maintain the video and audio synchronized during playback. The audio 
data is typically compressed as well, and the MPEG standard specifies an audio compression method such as MPEG 
Layer II, also known by the Philips trade name of "MUSICAM". 
55 An MPEG stream includes three types of pictures, referred to as the Intra (I) frame, the Predicted (P) frame, and 

the Bi-directional Interpolated (B) frame. The I or Intra frames contain the video data for the entire frame of video and 
are typically placed every 10 to 15 fram s. Intra frames provide entry points into the file for random access, and are 
generally only moderately compressed. Predicted fram s are encoded with reference to a past frame, i.e., a prior Intra 
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frame or Predicted frame. Thus P frames only include changes relativ to prior I or P frames. In general. Predicted 
frames receive a fairly high amount of compression and are used as references for futur Predicted frames. Thus, both 
I and P frames are used as references for subsequent frames. Bi-directional pictures include the greatest amount of 
compression and require both a past and a future reference in order to b encoded. Bt-dir ctional frames are not used 

5 for references for other fram s. 

After the I frames have been created, the MPEG encoder divides each I frame into a grid of a suitable size, e.g., 
16x16 pixel-squares, called macro blocks. The respective I frame is divided into macro blocks in order to perform 
motion compensation. Each of the subsequent pictures after the I frame are also divided into these same macro blocks. 
The encoder then searches for an exact, or near exact, match between the reference picture macro block and those 

10 in succeeding pictures. When a match is found, the encoder transmits a vector movement code or motion vector. The 
vector movement code or motion vector only includes information on the difference between the reference frame and 
the respective succeeding picture. The blocks in succeeding pictures that have no change relative to the block in the 
reference picture or frame are ignored. In general, for the frame(s) following a reference frame, i.e., P and B frames 
that follow a reference I or P frame, only small portions of these frames are different from the corresponding portions 

is of the respective reference frame. Thus, for these frames, only the differences are captured, compressed and stored. 
Thus the amount of data that is actually stored for these frames is significantly reduced. 

After motion vectors have been generated, the encoder then tracks the changes using spatial redundancy. Thus, 
after finding the changes in location of the macro blocks, the MPEG algorithm further reduces the data by describing 
the difference between corresponding macro blocks. This is accomplished through a math process referred to as the 

20 discrete cosine transform or OCT. This process divides the macro block into a suitable number of sub blocks, e.g., four 
sub blocks, seeking out changes in color and brightness. Human perception is more sensitive to brightness changes 
than color changes. Thus the MPEG algorithm devotes more effort to reducing color space rather than brightness. 

Each picture or frame also includes a picture header which identifies the frame and includes information for that 
frame. The MPEG standard also includes sequence headers which identify the start of a video sequence. Sequence 

25 headers are only required once before the beginning of a video sequence. However, the MPEG-2 standard allows a 
sequence header to be transferred before any I frame or P frame. The sequence header includes information relevant 
to the video sequence, including the frame rate and picture size, among other information. 

MPEG video streams used in digital television applications generally include a sequence header before every I frame 
and P frame. This is necessary to facilitate channel surfing between different video channels, which is an important 

30 user requirement. In general, when a user switches to a new channel, the video for the new channel cannot be displayed 
until the next sequence header appears in the stream. This is because the sequence header includes important infor- 
mation about the video sequence which is required by the decoder before the sequence can be displayed. If a sequence 
header were not included before each I frame and/or P frame, then when the user switched to a new channel, the 
video for the new channel possibly could not be immediately displayed, i.e., the video could not be displayed until the 

35 next sequence header. 

The sequence headers in an MPEG encoded stream include presentation timestamps or a time base within the 
encoded stream. Timestamps provide a user with a time reference relative to the beginning of a movie, enabling the 
user to accurately select or identify a sequence located midstream of the movie without having to reference the begin- 
ning of the movie. 

40 

Trick Play Streams 

In an interactive video-on-demand (VOO) or near-video-on-demand (NVOD) system, it is greatly desirable for the 
user to be able to selectively fast forward and/or fast reverse through the movie being watched. Thus, some video-on- 

45 demand systems include fast forward and fast reverse streams, referred to as trick play streams, for each movie. When 
the user desires to fast forward or fast reverse through a movie, the user selects the fast forward or fast reverse option. 
The respective fast forward or fast reverse trick play stream is then transferred to the user at the appropriate point 
where the user was watching, instead of the normal play stream, thus simulating a fast forward or fast reverse of the 
movie being watched. Typically, a single video stream, such as a movie, is encoded at different presentation rates to 

50 enable the video file to operate in fast forward or fast reverse speed in addition to the normal play presentation rate. 

Indexing 

Interactive video-on-demand syst ms which include trick play streams require methods for indexing between the 
ss normal play stream and the trick play streams, as well as for indexing between the trick play streams. In other words, 
when a user is watching a movie and choos s to fast forward for a period of time, a mechanism is needed for the video 
server to switch from the normal play str am to the appropriate point or fram in the fast forward stream. When the 
user then desires to resume watching at normal play speed, a mechanism is also needed for the video s rver to switch 
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from the frame b ing viewed in th fast forward stream to the appropriate point or fram in the normal play stream. 
Thus the video serv r must be able to determine the prop r positions within vid o files wh n a switch occurs in outputt ing 
a first video file at a first pr sentation rate to a second video fil at a second pr s ntation rate. 

On approach for ind xing between normal play and trick play streams includ s using look-up tables to index 

5 b tween the various streams. The look-up tables each include a plurality of indices which reference respective positions 
or I frames in the various streams. For xampl , index look-up tabl s can be generated using the MPEG presentation 
timestamps from the sequence headers of the normal play stream. 

One drawback to this approach is that the MPEG presentation timestamps may not always be available. For ex- 
ample, there is no requirement that the MPEG presentation timestamps be continuous, e.g., there could be breaks or 

10 gaps in the presentation timestamps. 

Another problem is that presentation timestamps are presentation-based. Thus, when a fast forward stream which 
is 5x fast is being played, the presentation timestamps do not advance 5x faster, but advance at the same rate as they 
do in a normal play stream. Thus in this method the server is required to perform computations on the presentation 
timestamps to determine the corresponding place in another stream. This increases the real-time processing burdens 

'5 on the media server. 

This approach also requires each decoder to have intelligence, and further requires the media server to interact 
with the decoder to accomplish stream switches. For example, when the user selects the fast forward or fast reverse 
option, in this method the decoder is required to provide information back to the media server of the respective pres- 
entation timestamp where the decoder stopped playing, as well as the presentation rate of the stream being played. 

20 The media server then uses this information to determine the appropriate presentation timestamp location to begin 
playing in the new stream. This requirement that the decoder interact with the media server to accomplish stream 
switches, as well as the computations required to be performed by the media server, increases the overhead of the 
system. The interaction between the media server and the decoder also requires that each decoder have intelligence, 
which increases the cost of each decoder. 

2* One such approach based on MPEG presentation timestamps is HP's "PictureNumber, PresentationTimeStamp, 

FileOffset" format for each table entry. Unfortunately, not all encoding formats are MPEG-based. Further accurate 
mapping between presentation rates can be accomplished only if the underlying assumption that the presentation rate 
is a constant ratio, i.e. one assumes the encoded video stream has a uniform frame rate, is true. Conversely, a uniform 
frame rate at all presentation rates disables techniques such as "scene fast forward". 

Therefore, an improved system and method is desired for efficiently indexing between normal play streams and 
trick play video streams in a video delivery. system. An improved system and method is further desired which reduces 
the processing burdens of the media server. 

SUMMARY OF THE INVENTION 

35 

The present invention comprises a system and method for indexing between related video streams in an interactive 
video delivery system. The interactive video delivery system preferably comprises at least one media server which 
stores video streams having different presentation rates. In the preferred embodiment, the system stores a normal 
play stream and one or more corresponding trick play streams. The trick play video streams are fast forward and/or 

*o fast reverse video streams. The present invention generates index look-up tables (ILUTs) between the normal play 
and trick play video streams which enable indexing between the streams, and the present invention uses these ILUTs 
to switch back and forth between the streams. 

In the preferred embodiment, the media server stores normal play video streams which are preferably compressed 
using any of various types of video compression methods, preferably an MPEG method. In creating the ILUTs, the 

« system first analyzes the normal play stream and creates a normal play time standard based on presentation times- 
tamps comprised in the normal play stream. The system then preferably creates an index look-up table for each of the 
normal play and trick play video streams using the normal play time standard. Each index table comprises an array of 
two-tuples, wherein the two-tuples are the normal play time standard and an index or offset into the respective stream. 
The index tables enable indexing between the streams. 

» During video delivery, the system of the present invention uses the respective index tables to switch back and forth 

between the normal play and trick play video streams. For example, when a user is viewing the normal play stream 
and desires to fast forward through the video stream, the media server examines the current normal play time and 
offset of the normal play stream being output in order to half the normal play stream at an appropriate point. The media 
server also uses the current normal play time to retrieve the appropriate offset in the fast forward stream index table. 

« This offset is then used to b gin play of the fast forward stream at the appropriate point or frame when the normal play 
stream is halted. When the user discontinues fast forwarding and selects normal play, the media server examines the 
current normal play time and offset of the fast forward stream being output to halt the fast forward stream at an appro- 
priat point. The m dia server also uses the current normal play time of the fast forward stream to retrieve the appro- 
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priate offset in the normal play stream index table. This offset is then used to begin play of the normal play str am at 
the location where the first forward stream was halted. Similar operations occur when the user fast reverses through 
the video stream. The pres nt inv ntion also provides a smooth transition between streams having different presen- 
tation rates by ensuring that stoppage and initiation of output of the different streams, i.e : , switching the output between 
5 the different streams, only occurs at well defined "random access" points. 

Therefore, the present invention fficiently allows indexing between normal play and trick play streams. The present 
invention creates a normal play time standard which is used as a common reference, thus simplifying the indexing 
process. This eliminates the requirement of any intelligence in the decoder and reduces.the processing requirements 
of the video server. 

10 

DESCRIPTION OF THE DRAWINGS 

A better understanding of the present invention can be obtained when the following detailed description of the 
preferred embodiment is considered in conjunction with the following drawings, in which: 

15 

Figure 1 illustrates a video delivery system including one or more media servers and one or more subscribers; 
Figure 2 illustrates the media server of Figure 1 ; 

Figure 3 is a block diagram illustrating the media server computer system of Figure 2; 

Figure 4 is a flowchart diagram illustrating generation of index look-up tables for normal play and trick play streams 
20 according to the present invention; 

Figure 5 illustrates index look-up tables for normal play and trick play streams according to the present invention; 
and 

Figure 6 is a flowchart diagram illustrating operation of the media server indexing between a normal play and trick 
play streams according to the present invention. 

25 

DESCRIPTION OF THE PREFERRED EMBODIMENT 
Video Delivery System 

30 Referring now to Figure 1 , a video server or video delivery system 30 for storing and transferring video streams 

is shown. The system 30 is preferably a video-on-demand (VOD) or near-video-on-demand (NVOD) system, or other 
type of video delivery system, which is capable of transferring or playing video or multimedia streams to one or more 
users, preferably a plurality of users. In the present disclosure, the term "video stream" is used to refer to a file or 
sequence of data for presenting a video display. The term "video stream" also includes a multimedia stream which 

3S includes both video and audio components. 

As shown, in one embodiment the video delivery system 30 comprises one or more media servers or video servers 
50 connected through a broadband network 40 to a plurality of subscribers 52. As discussed below, each media server 
50 preferably includes a general purpose computer system 60 (Fig. 2). The broadband network 40 is preferably a 
network suitable for multimedia content, such as an ATM (Asynchronous Transfer Mode) network. The subscribers 52 

40 preferably include display devices such as televisions, computers, etc. 

The media server 50 is capable of transferring or playing video or multimedia streams having different presentation 
rates. In the preferred embodiment, the system 50 is capabie of transferring or playing either a normal play stream or 
one or more trick play streams. The trick play streams may comprise one or more of a fast forward and/or fast reverse 
stream. Thus, in the present disclosure, the term trick play streams" refers to fast forward and/or fast reverse video 

*s streams, preferably compressed streams, which are generated from a normal play stream, and which have a different 
presentation rate than the normal play stream. 

As noted above, the normal play and trick play streams are preferably compressed video streams. An embodiment 
of the invention operates independently of the type or format of the video streams. Thus the video streams may be 
compressed in any of various types of formats, including MPEG-1, MPEG-2, Motion JPEG, QuickTime, etc. Further, 

50 an embodiment of the invention operates independently of the frame rate and other presentation characteristics. 

Figure 2 « Media Server 

Referring now to Figur 2, in this embodiment the media server or video server 50 comprises a computer system 
55 60. Figure 3 is a block diagram illustrating the components comprised in the media server computer system 60 of 
Figure 2. The media server computer system 60 includes various standard components, including one or more proc- 
essors, one or more bus s, a hard drive and memory. It is noted that Figure 3 is illustrativ only, and other computer 
architectures may be used, as desired. As shown, the computer system 60 includes at least one processor 80 coupled 
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through chips t logic 82 to a system memory 84. The chipset 82 includes a PCI (Peripheral Component Interconnect) 
bridge for interlacing to PCI bus 86. The computer system 60 includes a RAID (Redundant Array of Inexpensive Disks) 
disk array 90 or other storage media for storing the normal play streams and corresponding trick play streams. The 
computer system 60 may include ith r or both of an MPEG decoder 74 and MPEG encoder 76, which are shown 
5 connected to PCI bus 86. Th computer system 60 may also include video circuitry 88, as shown. 
- Referring again to Figur 2, th computer system 60 includes or is coupled to one or more digital storage or media 

. storage devices. For example, in the embodiment of Figure 2 the computer system 60 couples to a media storage unit 

, 62 through cable 64. The media storage unit 62 may be in addition to. or instead of, a disk storage system in the 
computer system 60. The media storage unit 62 includes one or more composite RAID drives for storing the normal 
10 play streams and corresponding trick play streams. Alternatively, the media storage unit 62 may instead include one 
or more CD-ROM drives and/or one or more Digital Video Disk (DVD) storage units, or other types of media, for storing 
digital video. The computer system 60 may also include one or more internal CD-ROM drives or may couple to one or 
more separate Digital Video Disk (DVD) storage units. The computer system 60 also may connect to other types of 
digital or analog storage devices, as desired. 
'5 The compressed normal play and trick play streams may be comprised on a storage media in the media server 

50, such as a RAID disk array, CD-ROM or Digital Video Disk (DVD). The media server 50 reads the respective normal 
play or trick play stream from the storage media and provides the data out to the one or more display units or viewers 
(subscribers) 52. The media server 50 may output the video data using various communication media, such as ATM 
; (Asynchronous Transfer Mode). ISDN (Integrated Services Digital Network), or via satellite. As noted above, the sub- 

■ . 20 scriber display units may comprise televisions, computer systems or other systems with a display screen for displaying 
video content. 

As mentioned above, media server 50 indexes or switches between normal play and trick play video streams, 
generally based on user selections. As discussed further below, the media server 50 generates index tables for the 
various streams and uses these tables to switch between the various streams. In this embodiment, the index table 
.25 generation and indexing functions are performed by the media server 50 in software, wherein the software is repre- 
sented by floppy disks 72. In another embodiment, the computer system 60 includes dedicated hardware which per- 
forms one or both of the index table generation and indexing functions. 

It is noted that the media server 50 may comprise two or more interconnected computers, as desired. It is noted 
that any of various types of video delivery systems may be used according to the present invention, as desired. 

30 

Figure 4 -Creation of Index Look-up Tables 

Referring now to Figure 4, a diagram illustrating generation of index look-up tables (LUTs) according to the present 
invention is shown. Here it is presumed that a normal play stream and one or more trick play streams are stored in the 
35 system. The different streams preferably encode the same content for presentation at different rates. 

As shown in step 102. server 50 receives or examines a normal play video stream or multimedia stream. As 
discussed above, the normal play stream comprises a stream of video data which is used to present a video sequence, 
such as a television segment or movie, onto a screen, such as a television or a computer system. In this embodiment, 
the normal play stream is a compressed stream, preferably an MPEG-2 compressed stream, although other types of 
*o compression may be used, as desired. Accordingly, the index LUTs are generated using the existing MPEG-encoded 
video stream. 

In step 104, server 50 analyzes timestamps within the stream. In this embodiment where the stream is an MPEG 
stream, the system analyzes the presentation timestamps from the sequence headers in the stream. As mentioned 
above, the presentation timestamps are used to provide a time base for the video sequence. 

« As discussed above, an MPEG encoded stream includes a plurality of I frames which are intracoded pictures, and 

a plurality of B and P frames which are intercoded frames. The I frames each contain video data for an entire frame 
of video and are placed periodically in the sequence. The P and B frames include change information relative to prior 
or subsequent frames. Each picture or frame also includes a picture header which identifies the frame and includes 
information for that frame. An MPEG encoded stream further includes one or more sequence headers which include 

so certain information regarding the video sequence, including the frame rate and the picture size, among other informa- 
tion. The sequence headers include presentation timestamps which indicate the play time of the video sequence. 

In step 106 server 50 maps the presentation timestamps to a "normal play time" (NPT) standard. Thus server 50 
defines a multimedia index based on the concept of NPT that can be associated with a 'position' within a multimedia 
title. Positions are defined to be equivalent between multimedia or video streams having different presentation rates 

S5 when th content pr sent at the r spective position is conceptually substantially equivalent. Henc , for video data, the 
position is defined to be equivalent when the same or substantially the same image in the sequence is being presented, 
allowing for differences in resolution and other encoding parameters than may be particular to th stream. 

In an embodim nt of the invention, NPT provides an indication of contextual position within a compressed video 
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stream, or any oth r multimedia fil , by assigning an increasing numeric valu to succeeding elements, e.g., frames 
or sequences, in the stream. As a result, NPT permits the location of a particular item of content within the video stream 
to be determined regardless of the presentation rate, encoding scheme or storage medium. 

In generating the normal play time standard, server 50 examines the presentation timestamps and keeps track of 

s the original or base pr sentatbn timestamp at the beginning of the movie. Server 50 then subtracts that base or original 
presentation timestamp from subsequent timestamps to determine the norma) play time values for the normal play 
stream, thereby compensating for any non-zero base presentation timestamp. In other words, in order to calculate 
normal play time for a given point in the normal play stream, the system subtracts the base presentation timestamp 
from a future presentation timestamp at the respective point or location in the normal play stream to determine the 

10 normal play time value for that location. 

In this embodiment, the normal play time (NPT) for a position in a multimedia stream is the time from the beginning 
of the title until the respective position when measured by presentation of the normal speed forward or normal play 
stream. Therefore, the concept of normal play time is used. Normal play time corresponds to the speed of the normal 
play stream and has a one to one correspondence with clock time. Thus, every second the normal play movie ticks 

is forward, normalplay time clicks forward one second. In a fast forward file or fast forward trick play stream, if the FF 
stream is 5x faster, normal play time is 5x faster as the user is watching. 

In general, any particular scene in the movie is identified by a normal play time. Thus, if a particular scene occurs 
at X minutes into the movie in normal play time, then this position or scene is referred to as or called X minutes. This 
particular scene is also located in any of the other trick play streams at X minutes normal play time. Thus in the fast 

20 forward and fast reverse streams, even though time is going by much faster, at X minutes normal play time the particular 
scene occurs. 

In step 108 server 50 creates an index look-up tables for each of the multimedia streams, i.e., for the normal play 
stream and each of the trick play streams. The index look-up table for the normal play multimedia stream comprises 
an index or array of two-tuples. The index look-up tables for a normal play, fast forward, and fast reverse stream are 
25 shown in Figure 5. As shown, each tuple comprises a normal play time value and a corresponding file offset within the 
stream. 

Note that the entries in each NPT index may be constrained by requirements of the encoding scheme. For example, 
some encoding schemes may only allow random positioning into the encoded stream at certain non-linear intervals. 
In the case of an MPEG2 transport stream, the "random access indicator" is set within the transport packet header to 

30 indicate the file offsets of the respective encoded data packets and resulting NPT indices are "randomly accessible". 

For the normal play stream, the normal play time entries comprise the normal play time values computed in step 
106. For the scaled streams, e.g., the fast forward and fast reverse streams, a scale factor is introduced into the normal 
play time values of the index look-up tables to compensate for the different presentation rates. Scaling of the presen- 
tation timestamps can be accomplished by multiplying the compressed presentation timestamp value by the ratio of 

3S the presentation rate to the normal presentation rate. 

It is noted that equivalent positions in multimedia streams having different presentation rates will have equal NPT 
values, although the actual time presentation from the beginning of the stream to that position will differ for the different 
streams. It is also noted that equivalent positions in multimedia streams having different presentation rates, although 
having equal NPT values, will have different byte offsets due to a presumptive difference in length of the streams having 

^0 different presentation rates. 

The index look-up tables specify indices or entries each based on a normal play time and a file offset to allow the 
multimedia server 50 to initiate or stop play at a particular normal play time point in the multimedia stream. The index 
look-up table indices also allow the multimedia server 50 to transfer to and between equivalent positions between 
streams of different presentation rate, i.e., between normal play and trick play streams. The index look-up table only 

is includes tuples representing valid positions for starting stopping, or transferring between the streams. 

The creation of the look-up tables is independent of any particular type of video compression or MPEG represen- 
tation. Hence, where MPEG compression is used the index look-up tables are created by scanning through the MPEG 
file, noting random access points in the MPEG file to compensate for presentation timestamp discontinuities, and then 
converting from the presentation timestamp in the MPEG file into the normal play time standard. Conceptually, each 

so index table comprises an array of normal play time vs. scenes, and any particular image in the movie can be identified 
by the normal play time value. As noted above, an index table is created for each presentation rate, e.g., fast forward, 
fast reverse, and normal play. Each of the offsets stored in the index table is an index from the normal play time to a 
byte offset in that MPEG file where the particular scene begins. 

Hence, server 50 us s a normal play time standard, inst ad of using timestamps in the video stream. As noted 

55 above, the presentation timestamps in an MPEG sequence are typically not always available. For exampi , there is 
no requirement that the timestamps be continuous, e.g., there could be breaks or gaps in the presentation timestamps. 
Therefore, unlike prior art methods, an embodiment of the invention do s not use the presentation timestamps as a 
basis for creating the ind x tabl s. Instead, an embodiment of the invention maps the presentation timestamps to a 
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normal play tim standard, and this normal play time standard is then used as a basis for creating the index tables. 
Flour 6 ■ Transferring Between Str ama 

Referring now to Figur 6, a flowchart diagram illustrating operation of the system of the present invention trans- 
ferring outputs between multimedia str ams having different presentation rates is shown. Here it is assumed that a 
current video or multimedia stream is being output from the media server 50, and the media server 50 has received 
user input indicating that a different stream should be output. For example, the media server may be providing the 
normal play multimedia stream, and the media server 50 receives user input of a fast forward or fast reverse selection, 
indicating that either the fast forward or fast reverse trick play stream should be output at the appropriate point. 

When user input is received indicating a desired change in the presentation rate, then in step 202 the media server 
50 finds a tuple in the index table of the current stream or file that contains an offset beyond the current output offset 
In other words, assuming that the current stream is playing and is at a certain point or offset within the stream, in step 
202 the media server 50 finds the tuple or entry in the index table of the current stream that contains an offset that is 
at or just beyond the current output offset. The current output offset is prefefebly provided by media file system (MFS) 
software executing in the media server 50. In step 202 the media server 50 receives this byte offset of the current 
output of the stream and searches the index table for the nearest offset greater than or equal to the byte offset of the 
current output of the stream. 

In step 204 the media server 50 schedules the current stream to terminate output at this offset determined in step 
202. Thus, in order to terminate a current stream being output, the media server 50 preferably finds the tuple for the 
nearest subsequent normal play time or nearest offset of the location currently being played and utilizes the associated 
offset of this type to terminate play of the current video stream at this offset. 

In step 206 the media server 50 determines the normal play time for the current stream. It is noted that the normal 
play time for the current stream may have been previously determined in step 202. In other words, having determined 
the nearest offset greater than the byte offset of the current output stream in step 202, the corresponding normal play 
time value in this tuple may be used as the normal play time for the current stream. 

In step 208 media server 50 finds the tuple in the index table of the new stream, i.e., the stream to be output with 
the nearest normal play time to the normal play time of the current stream. In step 210 the media server 50 uses the 
offset of the found tuple in step 208 to initiate output of the new stream at that offset. The output of the new stream is 
preferably initiated after the current stream terminates, wherein the current stream is scheduled to terminate in step 
204 as described.above. 

Therefore, initiation and termination of the output of a respective stream being output at a given normal play time 
is accomplished by finding the tuple in the respective index table for the nearest normal play time and utilizing the 
associated file offset as the point to initiate or terminate play of the stream. Transferring between different multimedia 
data streams having different presentation rates is accomplished by utilizing entries in each of the respective tables 
of the current stream and the new stream to be played to reschedule termination of output of the current stream and 
the beginning of play of the new stream. 

In sum, the present invention provides a system and method for indexing between normal play and trick play video 
streams. An embodiment of the invention examines the presentation timestamps in the sequence headers of the normal 
play stream and creates a normal play time standard which is used for all streams. The system then creates index 
tables or look-up tables for the stream. The index tables for the streams comprise normal play time values and corre- 
sponding offsets into the respective stream. During play, the video delivery system uses these index tables to intelli- 
gently jump or index between the normal play and trick play streams. This approach also permits non^onstant pres- 
entation rates such as scene forward or presentation rates based on content complexity. 

Other modifications may be used to generate a normal play time standard for the normal play stream without 
departing from the present invention. For example, NPT index LUTs can also be generated prior to encoding of the 
video streams by using frame numbers or sequence numbers. Alternatively, the NPT indices may be generated con- 
currently with the videocontent encoding. In either case, NPT indices can be generated by multiplying the frame number 
by the frame rate. This NPT position can then be associated with the file offset of the encoded frame. 

In yet another exemplary embodiment, the video content is encoded at a constant bit rate. Accordingly, NPT entries 
for some (possibly proper) subset of the random access point specified by the encoded video stream may be generated 
using the following equations: 

For forward presentations (i.e. positive presentation rate scale factor), the NPT value for each file offset at which 
a Random Access Point occurs may be calculated using the equation: 



NPT = (PresentationRateScaleFactor * FileBitOffset) / ConstantBitRate 
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Conversely, for revers presentations (i.e. negativ presentation rate seal factor), the NPT value for each file 
offset may be calculated using the quation: 

5 NPT = Tola IN PT + ((PresentationRateScaleFactor * FileBitOff set) / ConstantBitRate) 

Wherein: 

PresentationFteteScaleFactor = ratio of presentation rate with respect to normal presentation rate (e.g. a value of 
10 7 indicates 7x fast forward, a value of -5 indicates 5x fast reverse); 

FileBitOff set = the number of bits from the beginning of the encoding to the file offset specified as a random access 
point; 

ConstantBitRate = the constant bit rate at which the encoding is intended to be played; and 
Total NPT = the total time duration of normal speed presentation (i.e. what would be commonly though of as the 
is length of the movie). 

Although the system and method of the present invention has been described in connection with the described 
embodiments, it is not intended to be limited to the specific form set forth herein, but on the contrary, it is intended to 
cover such alternatives, modifications, and equivalents, as can be reasonably included within the scope of the invention . 

20 

Claims 

1. A computer-implemented method for indexing a first and a second related video stream having different presen- 
ts tation rates, the method comprising the step of creating index look-up tables (LUTs) for each of said related video 

streams using a normal play time standard, wherein each of said index LUTs includes a plurality of entries com- 
prising a normal play time value and a corresponding offset into the respective video stream. 

2. The method of claim 1 , wherein the video streams include a normal play stream, wherein said creating index look- 
so up tables for said first and second video streams includes: 

receiving the normal play stream, wherein the normal play stream includes a plurality of timestamps; and 
mapping said plurality of timestamps to said normal play time standard; 

35 wherein said creating index look-up tables for said first and second video streams uses said normal play 

time standard. 

3. The method of claim 2, wherein said video streams having different presentation rates comprise MPEG compressed 
streams; 

40 wherein said mapping said plurality of timestamps to a normal play time standard comprises examining 

sequence headers in said MPEG compressed normal play stream for said plurality of timestamps. 

4. The method of claim 1 , wherein said video streams-include trickplay streams including a fast forward stream and 
a fast reverse stream. 

45 

5. The method of claim 1 , wherein said normal play time has a one to one correspondence to clock time. 

6. A computer-implemented method for transitioning between a first and a second related video stream having dif- 
ferent presentation rates, the method comprising: 

so 

creating index look-up tables (LUTs) for each of said related video streams using a normal play time standard, 
wherein each of said index LUTs includes a plurality of entries comprising a normal play time value and a 
corresponding offset into the respective stream; 
transferring video data from said first video stream; and 
ss switching between said first video str am and said second video stream using said index look-up tables. 

7. The method of claim 6, wher in the video streams include a normal play stream, wherein said creating index look- 
up tables for said first and second video streams includes: 
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receiving the normal play stream, wherein the normal play stream includes a plurality of timestamps: and 
mapping said plurality of timestamps to said normal play time standard; 

wherein said creating index look-up tables for said first and second video streams uses said normal play 
5 time standard. 

8. The method of claim 7, wherein said video streams comprise MPEG compressed streams; 

wherein said mapping said plurality of timestamps to a normal play time standard comprises examining 
sequence headers in said MPEG compressed normal play stream for said plurality of timestamps. 

w 

9. The method of claim 6, wherein said switching between said first and second streams includes; 

determining the normal play time of said first video stream; 

determining an offset in the second video stream based on the normal play time of the first video stream; and 
15 initiating output of the second video stream at said determined offset in the second video stream. 

10. The method of claim 9, wherein said determining an offset in the second video stream based on the normal play 
time of the first video stream comprises; 

20 finding an entry in an index look-up table of said second video stream having a normal play time value close 

to the normal play time of the first video stream; and 

determining the offset in said found entry in said index look-up table of said second video stream. 

11. The method of claim 9, wherein said switching between said first and second video streams further comprises 

25 

determining an entry in an index table for the first video stream that contains an offset beyond an offset of data 
currently being output from the second video stream; and 

scheduling output of the current stream to terminate at said offset beyond the offset of data currently being 
output from the first video stream. 

30 

12. The method of claim 6, wherein the video streams having different presentation rates include trickplay streams 
including a fast forward stream and a fast reverse stream. 

13. The method of claim 6, wherein said normal play time has a one to one correspondence to clock time. 

35 

14. A computer-implemented method for transitioning between a first and a second related video stream, each said 
video stream having different presentation rates, the method comprising; 

transferring video data from said first video stream; 
40 referencing an index look-up table (LUT) for each said video stream, wherein each said LUT includes a plurality 

of entries comprising a normal play time (NPT) value and a corresponding offset into the respective video 
stream; 

switching from said first video stream to said second stream using the index LUTs; and 
transferring video data from said second video stream. 

45 

15. The method of claim 14, wherein the video streams include a normal play stream, wherein said creating index 
look-up tables for said first and second video streams includes: 

receiving the normal play stream, wherein the normal play stream includes a plurality of timestamps; and 
50 mapping said plurality of timestamp - r o said normal play time standard; 

wherein said creating index look-up tables for said first and second video streams uses said normal play 
time standard. 

ss 16. The method of claim 1 5, wherein said video streams comprise MPEG compressed streams; 

wherein said mapping said plurality of timestamps to a normal play time standard comprises examining 
sequence headers in said MPEG compressed normal play stream for said plurality of timestamps. 
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17. The method of claim 14, wherein said switching between said first and second streams includes: 

determining the normal play time of said first video stream; 

determining an offset in the second video stream based on the normal play time of the first video stream; and 
5 initiating output of the second video stream at said determined offset in the second video stream. 

18. The method of claim 17, wherein said determining an offset in the second video stream based on the normal play 
time of the first video stream comprises: 

10 finding an entry in an index look-up table of said second video stream having a normal play time value close 

to the normal play time of the first video stream; and 

determining the offset in said found entry in said index look-up table of said second video stream. 

19. The method of claim 17, wherein said switching between said first and second video streams further comprises: 

1$ 

determining an entry in an index table for the first video stream that contains an offset beyond an offset of data 
currently being output from the second video stream; and 

scheduling output of the current stream to terminate at said offset beyond the offset of data currently being 
output from the first video stream. 

20 

20. The method of claim 14, wherein the video streams having different presentation rates include trickplay streams 
including a fast forward stream and a fast reverse stream. 

21. The method of claim 14, wherein said normal play time has a one to one correspondence to clock time. 

25 

22. A video server which provides video streams having different presentation rates, wherein the video server indexes 
between said video streams having different presentation rates, the video server comprising: 

video memory configured to store the video streams having different presentation rates; 
30 an index look-up table (LUT) for each of said video streams, wherein the index look-up tables are based on 

a normal play time standard, wherein each of said index look-up tables includes a plurality of entries comprising 
a normal play time value and a corresponding offset into the respective stream; 

one or more output ports coupled to said video memory for transferring video data from a video stream; and 
a switch coupled to said video memory and said memory, and configured to switch between said video streams 
35 at said one or more output ports, wherein said switch uses said index look-up tables in switching between said 

video streams. 

23. The video server of claim 22,wherein the video streams include a normal play stream, the video server further 
configured to create indices for said LUTs by examining the normal play stream which includes a plurality of times- 

•to tamps and by mapping said plurality of timestamps to said normal play time standard. 

24. The video server of claim 22, wherein said video streams includes MPEG compressed streams, and said LUT 
generator maps said plurality of timestamps to said normal play time standard by examining sequence headers 
in said MPEG compressed normal play stream for said plurality of timestamps. 

45 

25. The video server of claim 22,f urther configured to determine the normal play time of a current stream being played, 
to determine an offset in a new stream based on the normal play time of the current stream, and to initiate output 
of the new stream at said determined offset in the new stream. 

50 26. The video server of claim 25, further configured to locate an entry in an index look-up table of said new stream 
having a normal play time value close to the normal play time of the current stream, and to determine the offset 
• v in said found entry in sai«d index look-up table of said new stream. 

27. The video server of claim 25, further configured to determine an entry in an index table for the current stream that 
55 contains an offset beyond an offset of data currently being output from the current stream and to schedule output 

of the current stream to terminat at said offset beyond the offset of data currently being output from the current 
stream. 
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28. The video server of claim 22, wherein th video streams having different presentation rates include trickplay 
streams including a fast forward stream and a fast r verse stream. 

29. The video server of claim 22, wherein said normal play time has a one to one correspondence to clock time. 

30. A computer-readable storage media for operating in a computer system, the computer system including a central 
processing unit and memory, wherein the computer system stores video streams having different presentation 
rates, wherein the computer-readable storage media includes a substrate having a physical configuration repr - 
senting data, the storage media comprising: 

an index look-up table creation program configured to create index look-up tables for each of said video streams 
using a normal play time standard, wherein each of said index look-up tables includes a plurality of entries 
comprising a normal play time value and a corresponding offset into the respective stream; 
avideo stream switching program configured to switch between output of a current stream and a new stream 
using said index look-up tables. 

31 . The computer-readable storage media of claim 30, wherein the video streams include a normal play stream, where- 
in said index look-up table creation program includes: 

an examining program configured to examine the normal play stream, wherein the normal play stream includes 
a plurality of timestamps; and 

a mapping program configured to map said plurality of timestamps to said normal play time standard; 
wherein said index look-up table creation program uses said normal play time standard. 

32. The computer-readable storage media of claim 30, wherein said video stream switching program includes: 



a program configured to determine the normal play time of the current stream; 

a program configured to determine an offset in the new stream based on the normal play time of the current 
30 stream; and 

a program configured to initiate. output of the new stream at said determined offset in the new stream. 

33. The computer-readable storage media of claim 32, wherein said program for determining an offset in the new 
stream based on the normal play time of the current stream comprises- 

3S 

a program configured to find an entry in an index look-up table of said new stream having a normal play time 
value close to the normal play time of the current stream; and 

a program configured to determine the offset in said found entry in said index look-up table of said new stream. 

*o 34. The computer-readable storage media of claim 32, wherein said video stream switching program further comprises: 

a program configured to determine an entry in an index table for the current stream that contains an offset 
beyond an offset of data currently being output from the current stream; and 

a program configured to schedule output of the current stream to terminate at said offset beyond the offset of 
45 data currently being output from the current stream. 

35. A video subscription system for receiving and displaying video streams having different presentation rates from a 
video server, wherein the video server indexes between said video streams having different presentation rates, 
the video subscription system comprising: 



55 



a video display device configured to display the video streams having the different presentation rates; and 
a controller coupled to said video display device and configured to cause said video server to switch between 
said video streams by using an index look-up table (LUT) for each of said video streams, wherein the index 
look-up tables are based on a normal play time standard, wherein each of said index look-up tables includes 
a plurality of entries comprising a normal play time (NPT) value and a corresponding offset into the respective 
stream, and wherein said video streams include a normal play stream and said entries are created by exam- 
ining the normal play stream which includ s a plurality of timestamps and by mapping said plurality of times- 
tamps to said normal play time standard. 
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